Doctoral Thesis Techniques for Improving Voice Conversion Based on Eigenvoices

Voice conversion (VC) is a technique for converting a source speaker’s voice into another speaker’s voice without changing linguistic information. As a typical approach to VC, a statistical method based on Gaussian mixture model (GMM) is used widely. A GMM is trained as a conversion model using a parallel data set composed of many utterance-pairs of source and target speakers. Although this framework works reasonably well, the converted speech quality is still insufficient and a training process of the conversion model is less flexible. Eigenvoice conversion (EVC) is an effective method for making the training process more flexible. An eigenvoice GMM (EV-GMM) is trained in advance with multiple parallel data sets consisting of the single pre-defined speaker and many pre-stored speakers. Then, a conversion model for a new speaker is flexibly built by adapting the EV-GMM using a few arbitrary utterances of the new speaker. Two main frameworks have been proposed based on EVC: 1) one-to-many EVC, which allows the conversion from a single source speaker’s voice into an arbitrary target speaker’s voice; and 2) many-to-one EVC, which allows the conversion in reverse. Although these frameworks achieve much higher flexibility than the traditional VC, there are still remaining limitations in building the conversion model between an arbitrary speaker-pair. In addition, the conversion performance of the EVC is significantly degraded because the EV-GMM captures acoustic variations among the pre-stored target speakers. To make VC applications more practical, it is indispensable to improve the conversion performance and develop a more flexible training framework. ∗Doctoral Thesis, Department of Information Processing, Graduate School of Information Science, Nara Institute of Science and Technology, NAIST-IS-DD0761009, February 4, 2010.